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Statistical Properties of Weighted Binary 
Random and Pseudo-Random Sequences 

0.0 Introduction 

The kind of problem discussed in this final report is illustrated in 
figure 6-1; 

An n-stage binary shift register accepts the input binary random or 
pseudo-random sequence. The sequence is shifted_one-stage-t-o-the - rfgh't'¥t 
each clocking time.' The values in the stages of the register are multiplied 
or weighted by the numbers {a-j, a2» a n > . The resulting products 

are summed to provide the output from the system. If the input is a pseudo- 
random sequence the shift register may also serve as the sequence generator 
by connecting it as a feedback shift register. 

This problem was discussed in detail in an earlier report [1], which - 
includes an extensive bibliography. The present report presents a number 
of extensions of [1]. 

First, the input is assumed to be pseudo-random, and the weights are 
0 or 1 . We consider the possible forms of the probability density function 
of the output, and some of its moments. 

i ^ 

Secorid, the input is assumed to be either random or pseudo -random, and 
we consider the autocorrelation function and the power spectral density of 
the output.’ 

Third, we assume that n -> » , and consider some fundamental properties 
of infinite sums of random variables. 

Fourth, the present status and initial experiments in a new communications 
laboratory are discussed. It is shown how this laboratory will be used to 
study many of the concepts considered above. The design and testing of a 
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binary nonrecursive digital filter is discussed in detail. 

Fifth, some unexpected relations to some important basic mathematical 
relations are obtained. It is shown that such sequences as successive 
partition numbers, and the Fibonacci sequence, can be generated by digital 
filter circuits. 


0.1 Applications 

The problems studied here have a wide range of applications. First, 
we are concerned with the generation of random number sequences with different 
statistical properties. Such sequences are used in communications systems, 
system identification, equipment testing, and potentially in a great many 
other areas where random signals are required. These applications will 
continue to grow in number and importance as the trend towards digitalization 
of networks and systems continues. 

Second, we are concerned with the effect of digital filters on random 
sequences. In some sense this is related to the first problem. In this 
case, however, the emphasis is on filtering. Digital filtering is rapidly 
becoming a very important approach to filtering. The effect which such 
filters have on random sequences is of great importance. The techniques 
developed here can be used to design filter to have desired effects on 
sequence autocorrelation functions. 

Third, we have done some hardware design and development work on binary 
nonrecursive filters, which will be useful in implementing such filters. 

Finally, a number of the ideas and hardware which have evolved from 
this study have led directly to applications in education. Material 
emanating from our work is introduced in undergraduate courses in probability 
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theory, (AM 108) and communications theory (EE 141), as well as in graduate 
courses in the same areas. A paper [17] has recently been prepared describing 
many of the ideas discussed here in an educational context. In addition a 
pair of courses in digital filtering, one at the undergraduate and one at the 
graduate level, based largely on our contract work, are being prepared for 
the winter of 1971. Thus the stimulus to our academic program has been both 
clear and direct. 



1.0 Forms of Probability Density Functions 
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Some recent papers [2] - [6] have discussed the problems of the sum 
of weighted digits in an n-stage feedback shift register connected to 
generate maximal length sequences. In particular the papers by Lindholm 
[2] and Davies [5] have been concerned with the sum over m unity-weighted 
stages. They consider the problem of the statistics of the pseudo-random 
variable 

m-1 

V'i - I a k . 0-1) 

1 k=0 K 1 

where a. is the i^* 1 value (0 or 1) of the m-sequence having period L - 2 n - 1 . 

f 

n is the number of stages in the shift register feedback loop. The primary 
interest of these two papers is in the first few moments of w. . Davies also 
presents graphical results of the actual distributions of w. for n = 5 and 
1 m <_ 31 . 

In this study the sum (1-1) is considered over any m stages that need 
not necessarily be successive. The probability density functions (PDF) of 
w* are obtained in a straight-forward manner for different combinations of 
m stages. These PDF’s are compared and found to be consistent with the 
results of reference [5] . The problem is considered for a particular case 
but the approach is applicable to other cases. 

1.1 The Probability Density Function 

Consider the 5-stage feedback shift register with generating polynomial 
in the delay operator D of the form D 5 @ D^ » D° , where D 1 stands for i 
units of delay' and © stands for modulo-2 .addition. A shift register with five 
'stages in the feedback loop and 26 additional stages available for summation i 
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shown in Figure 1-1 . We seek the distribution of the analog sum: 

S k = D 1 + D 2 + D 3 + D 4 + D 5 + D k , 1 k £ 31 (1-2) 

This is the sum of the first five' stages plus any other one of 31 

stages. (It is not of interest to consider more than 31 stages since the 
sequence has period 31). Consider first the case where k = 6 in equation (1-2). 

S g = D 1 + D 2 + D 3 + D 4 + D 5 + D 6 

= D 1 + D 2 + D 3 + D 4 + D 5 + (D 1 © D 3 ) (1-3) 

The second equality follows from the generating polynomial* 

(D 5 © D 2 = D° implies D 6 = D 1 © D 3 ) 

We now apply a method introduced in reference [6]. Group the terms in 
equation (1-3) according to the order of the delay. 

S 6 = [D 1 + D 3 + (D 1 © D 3 ) ] + D 2 + D 4 + D 5 (1-4) 

The last three terms and the bracketed term in (1-4) are independent 
if we assume the all -zero state in the five-stage feedback shift register 
for computational purposes. Hence their PDF's can be convolved to find the 
PDF of Sg . The PDF of the bracketed term is found from the following table: 

D 1 D 3 D 1 © D 3 D 1 + D 3 + (D 1 © D 3 ) 


0 0 0 
0 1 1 
1 0 1 
1 1 0 


0 

2 

2 

2 


From the above table the PDF of the bracketed term can be written as 
the sequence 


{1/4 0 3/4} 

where the order of the probability terms refers to sum values of 0, 1 and 
2 from left to right. Again assuming the all-zero state for computational 




Modulo-2 Adder 


Figure 1-1. Sum of 6 Pseudo-Random Sequences 
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purposes the PDF for each of D^, D 4 and is {1/2 1/2) : 

The -PDF of the sum S g is thus the convolution: 

(1/4 0 3/4} * {1/2 1/2} * {1/2 1/2} * {.1/2 1/2} = 

1/32 { 1 3 6 10 9 3 } 

and, neglecting the all -zero state, this becomes: 

1/31 { 0 3 6 10 9 3 } 

If we repeat this procedure for Sy and Sg we get the same PDF as for S g . 
If, however, v/e consider S g we obtain: 

S 9 = D 1 + D 2 + D 3 + D 4 + D 5 + D 9 

« D 1 + D 2 + D 3 + D 4 + D 5 + D 6 © 0 4 

= [ D 1 + D 3 + D 4 + (D 1 © D 3 6 D 4 )] + D 4 + D 5 (1-5) 

With PDF 

1/31 { 0 2 7 12 7 2 1 } 

The important difference between equations (1-4) and (1-5) is that in 
equation (1-4) there are two interdependent stages (1 and 3) whereas in 
equation (1-5) there are three interdependent stages (1, 3 and 4). We obtain 
a different PDF for different numbers of interdependent stages. The five 
possible types of PDF are given in Table 1-1. The first moment and the 
second through fifth central moment are also given. 

There are a total of 31 ways to obtain all the PDF's. The number of 
ways of obtaining each type is simply the combination of the 5 basic stages 
taken r at a time where r is the number of interdependent stages, r = 1 
corresponds to the case where k = 1 , 2, 3, 4 or 5. In this case the weighting 
is essentially 2 rather than 1 for one of the 5 stages. 

This basic approach can be applied to any combination of m stages. 

If m = 6 it is only necessary to group the terms appropriately and count 
the number of interdependent terms z<The distribution is given in Table 1-1. 



Number of 


Interdependent 

Stages 

Ways to 
Obtain PDF - 

PDF- (x 31) 
0 12 3 

>. 

4 

5 ~ 

-J 

_6 

Jh 

J 2 

y 3 

_^4 

_^5 

1 

5 

0 

4 7 8 

7 

4 

1 

3.097 

1.764 

0.360 

6.956 

4.423 

2 

10 

0 

3 6 10 

9 

3 

0 

3.097 

1.248 

-0.267 

3.614 

-1.638 

3 

10 

0 

2 7 12 

7 

2 

1 

3.097 

1.248 

0.508 

4.862 

5.427 

4 

5 

0 

1 10 10 

5 

5 

0 

3.097 

1.248 

0.508 

3.314 

2.305 

5 

1 

0 

0 15 0 

15 

0 

1 

3.097 

1.248 

0.508 

3.314 

6.176 


= E (S jP 

w r = E a \ - E c^)] r ), r = 2,3, 4, S 


TABLE 1-1 
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Also, of course, tables equivalents 1-1 can be obtained for any n and m 
following the basic procedure outlined above. 

1.2 Moments 

It is shown in reference [5] that the first moment of a sum over m 
stages is: 

E, = m (L + 1) (1-6) 

i -“21: 

which is called m-j in Table 1-1. 

E-j is not dependent on which m stages are selected. Hence it is 
not surprising that, in agreement with (1-6), all the distributions in 
Table 1-1 are found to have E.j = 96/31 = 3.097. 

The variance, as obtained from equations (2) and (3) in reference 
[5] is: 

a 2 = v 2 = ro(l+l ) (L-m) (1-7) 

4L 2 

This result is based on an assumption that the summed terms are from 
successive stages. For m = 6 and L = 31 , equation (1-7) gives c = 1.248. 

As Table 1-1 indicates, this is in agreement with our results for all values 
of k except k = 1, 2, 3, 4, 5 (one interdependent stage). Hence the 
variance depends only on the number of stages summed, as long as they are 
separate stages, and not on which particular stages are selected. That 
is, they need not be successive. A study of the mathematics in reference [5], 
which leads to the expression for the second moment, indicates that this 
result is reasonable. The summations involved are essentially dependent 
only on the number of terms in the sums and not on the order of summation. 
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Similarly, the third moment y^ is the same for 3, 4, and 5 
interdependent stages, and the fourth moment y^ is the same for 4 
and 5 interdependent stages. In general, for this case, the 
moment (or central moment) is the same for all those cases where 
there are at least r interdependent stages. A number of other cases 
of n (for 4 <_ n £ 10) were partially checked; it was found that the 
rule in ' the~"preceding sentence holds for all cases investigated. 

THe results indicated in this chapter were published by Healy 
[7]. This publication 'was responded to by Davies [8], who indicates 
an alternative approach to obtaining the PDF, through the use of 
transforms. 



2.0 Autocorrelation and Spectral Properties 
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This section is concerned with the autocorrelation function of 
weighted and summed pseudo-random and random sequences. Consider the 
circuit shown in Figure 0-1. The v outputs of the register stages are weighted 
by (a-j , a 2 , •••) and added to yield an output from the analog adder which 
is the pseudo-random or random sequence of interest here. 

The PDF pseudo-random outputs has been studied in some recent papers 
([2] - [6]). In this section we restrict our interest to the autocorrelation 
function and power spectral density of the output. 

The autocorrelation function of a sequence can be considered from two 
viewpoints. First, it can be considered as discrete if we center our inter- 
est on an entire clocking period at a time. This is the viewpoint which 
is of interest, for example, to the computer user who is generating pseudo- 
random or random numbers. Alternatively, we may consider the clocking time 
to be unknown. This viewpoint is commonly of interest to communi cations 
engineers. We consider first the discrete viewpoint. 

2.1 Discrete Autocorrelation of Pseudo-Random Processes 


Let a-j = 1 and a. = 0 for all i j* 1 . Then the output is a binary 
sequence that we write as: 

[b-j , b 2 j b^ , *** * b^ **• * b 2 n— 1 » b-j , b 2 , ***] 

For this correlation study it is convenient^ to assume that the b^ take 
values +1 and -1. Since the sequence is periodic, we have + 2 n - 1 = ^k 
The discrete autocorrelation function is defined as: 


1 


2 n -l 


2 n -l k=l 


b k b k+m 


, m = 0, 1 , 2 


R-j (m) = 


(2-1) 
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Consider first the pseudo-random case. It is well known [9] that the 
autocorrelation function of a maximal -length pseudo-random sequence is: 


R-j (m) = 


1 

1 

2 n -l 


, m = 0 
, m f 0 


(2-2) 


for | m | < 2 n -l and that R-j (m) is periodic with period 2 n -l . 

Consider now the autocorrelation of the output of the system shown 
in Figure 0-1. We assume the device is connected as a feedback shift register 
of length n stages for pseudo-random number generation, and there are L 
successive non-zero weights. That is, = 0 for i > 1. Also assume L < n. 

Then an output value, which we might call the "k tb " output, is: 

( a L b k + a L-l b k+l + + a l b k+t-l^ 

and a value m clock periods later is . - 

( a L b k+nt + a L-l b k+l+m + ••• + a l b k+L-Hm ) * 

Hence, the autocorrelation function is: — — 

^2^-1 

= ^ ^ a L b k * a L-l b k+l + *** + a l b k+L-l^’ ^ 2 ‘ 3 ^ 

(a L b k-Fm + a L-l b k+l+m + ••• + a l b k+L-l -n^ 

In forming the product indicated in (2-3), we collect terms having the 
same difference in the indices (subscripts) of b. Equation (2-3) thus becomes: 

2 n _i 

Rl( m ) = __1_ (a L 2 b k h k+m + a L 2 r b^b^; 1+m +“T. . + a^b k+L-1 b k+L . 12 +m 

+ a L a L-l b k+l b k+m + ••• + a 2 a l b k+L-l b k+L-2-Hn 


+ . • «, • 


* a L a l b k+L-l b k+m) 


(2-4) 



We assume the process is stationary with respect to clock intervals. 
A comparison of equations (2-1) and (2-4) then tells us that each term in 
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the first row in (2-4) is proportional .to R^(m), each term in the second 
row is proportional to R^(m-l), etc. Hence equation (2-4) reduces to: 

R L (m) = (a L 2 + + ... + a-, 2 ) R ] (m) + 

( a L a L_l + a L-l a L-2 + •*• + a 2 a l^ R i( m ~i) + 

... + (a-ja^) R.j (m-L+1 ) 

= l l a.a, R-, (m+r) , r<L (2-5) 

r=0 i=l 

This relation holds for jm ] < 2 n -l . For other values of m we need 
only note that R(m) is periodic with period 2 n -l . This arises from the fact 
that the original pseudo-random sequence has this same period. 

Although the relation (2-5) has been derived for a specific kind of input, 
it can be easily generalized for any input autocorrelation R x (m) and output 
autocorrelation R y (m). 

L-l L-r 

R y (m) = I l a i a i+r R x( m - r) ( 2 ~ 6 ) 

where L again stands for the number of weights. 

Let us consider two examples. First, let a^-l for 1 <_ i <^L, 3^=0 for 
i > L. That is, we assume unit weights for all L stages. Equation (2-5) 
then becomes: 

L-l 

R. (m) = l (L-r) R, (m + r) , r < L (2-7) 

r=0 1 

This result is plotted in Figure 2-1 for n = 4 and L = 1, 3 and 6. 

As a second example let us consider binary weighting; that is, let 
a.j = 2 1 ” for 1 < i <_ L and a^ = 0 for i > L. (It was shown by Davies [3] and 
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Healy [4] that this weighting leads to a uniform probability distribution). 
Substitution of = 2 1 ” into equation (2-5) leads to: 

L-l L-r • i j i 

' ‘ Mm) = l l 2 1 " 1 2 1 ' 1 r R,(m+r) 

L r=0 i=l 1 " 

L-l L-r • n 

* l 2 l 4 1 ' 1 Mmjr), 

r=0 i=l 

L-l r „ 

= y 2 r 4-1 R n (m+r) , r < L (2-8) 

r=0 3 ' 

The last form follows from the fact that the inner sum on the next to 
the last line is just a truncated geometric series. Equation (2-8) is 
plotted in Figure 2-2 for n = 4 and L = 4. 

The results obtained above apply to random as well as pseudo-random 
inputs. If the process is binary (taking values 1 and -1), the auto- 
correlation function can be defined as: 

R.j (m) = 1 , m * 0 (2-9) 

0, m f 0 

Other non-periodic (purely random) autocorrelations may be used in 
equation (2-5). 

2.2 A Convolution Approach 

In the analysis above we were interested in the relation of the output 
autocorrelation given the autocorrelation of a known input to a specified 
system. The system actually is a non-recursive digital filter, that is, a 
digital filter that does not use past output values to obtain subsequent 
outputs (outputs do not "recur".). Douce [10] has pointed out the analogy 
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with the problem of autocorrelation functions of signals into and out of 
continuous filters. As Papoulis [11] shows: 

R yy (t) « R X)? (t) * h* (~ t ) *h(t) (2-10) 

where R (t) is the input autocorrelation function to a filter, R (x) 

xx yy 

is the output autocorrelation function 5 h(x) is the impulse response, the 
asterisk operators denote convolution, and the asterisk superscript complex 
conjugation. 

In analogy with the R^(m) above, the output autocorrelation of non- 
recursive digital filter shown in Figure 1, can be obtained from: 

R L (m) = R-j (m) * h(m) * h(-m) (2-11) 

where h(m) is the discrete system analog to an impulse response. It is 
the response of the system to a sequence {1 0 0 0 ••• }, and it is 

simply equal to the sequence of weights { a-j ag ... a. ... a^ } . It is 
not necessary that the input autocorrelation R-j (m) in equation (2-11) have 
the same form as that given in equation (2-2). In fact it may have any 
form. 'If the form of R^ (m) is not simple, it may be more convenient to 
apply equation (2-11) rather than equation (2-5). 

Equation (2-11) suggests the hardware implementation shown in Figure 
2-3. This device may be attractive as a pedagogical tool to show the effect 
of filters on autocorrelation functions. The shift register on the left is 
easy to build if the autocorrelation is binary, such as { 1 0 0 0 

Then the shift register on the left will also be binary. But the shift 
register on the right must be m-ary where m may be quite large depending 
on the bounds put on the input and the weights. If the input is not binary, 
neither shift register can be binary. The major difficulty in building a 
device such as that shown in Figure 2-3 appears to be in the m-ary shift 
register. 



Input 


Autocorrelation 



2 


5 



Figure 2-3. Hardware Implementation of an 

Autocorrelation Function Generator 












2.3 Synthesis of Autocorrelation Function 
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Consider the basic circuit of Figure 0-1. Given an input autocorrelation 
R (m), specify the weights a. necessary to obtain a desired output auto- 

X I 

correlation R (m). This is the synthesis problem. 

This problem is solved through use of equation (2-6). Substitution 
of the first L integers { 0, 1, 2, ...» L } into (2-6) yields L independent 
simultaneous algebraic equations in the L unknowns where 1 £ i <_ L. 

Other equations which might be written, for negative m, are not independent 
because the autocorrelation is an even function. Hence we have L equations 
in L unknowns, and we can therefore find the required a.. . 

Of course not all autocorrelations can be synthesized for a given input 
autocorrelation. For example, if we let the input autocorrelation be 

R x (m) » 1, m=0 (2-12) 

0, m f 0 , 

then it is not possible to select a^ and a 2 obtain an autocorrelation of 
the form: 

R y (m) = 1. m = 0 or + 1 (2-13) 

0, m f 0 and m f + 1 

To see this let us carry out the synthesis operation. Equation (2-6), 
for L = 2, becomes 

V m) = r l Q a i a i+r R x (m — r) 

= a-, 2 R x (m) + a 2 2 R x (m) + a^^fm + 1) (2-14) 

R y (0) = ft] 2 + a 2 2 
R y (l) = a-,a 2 


(2-15) 
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If we substitute Ry(0) = 1 and R^O) = 1 into equations (2-15) 
we find that this pair of equations has no real solution. Solving (2-15) 


for a ? yields: . 

( R y (0) + / R y 2 (0) " 4R y Z (D J 


(2-16) 


It is clear that a necessary -condition' for a real a^ is 


R „0) 


W' 1 1 


(2-17) 


The equality in (2-17) gives the largest possible ratio of R^O) to 

R (0). As (2-16) suggests, this corresponds to uniform weighting (a, = 
y i y /-Y~ 

It seems reasonable, though it has not been proven here, that uniform weighting 

should lead to the autocorrelation function with the largest possible relative 

A 

values for m f 0. Vie also note that uniform weights results in an auto- 
correlation which falls off linearly with m. (See equation (2-7) and 
Figure 2-1.) If the above conclusion about relative values is correct, then 
any weighting*which is not uniform should result in an autocorrelation 
fall-off which is more rapid than a linear function of m. This phenomenon 
is illustrated on the n ght _$i de_o.f_.E-i gure-2-2^ fo r b inary weighting.- 

2.4 Continuous Autocorrelation Functions 


In this section we consider the case where the clocking ‘reference’ is 
not known. Then the autocorrelation function of a pseudo-random signal 


is [9]: 


R(t) = 0 


iiL i _ 1 


It < T 


(2-18) 


, T < r < (P-l)T 
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where p = 2 n -l , and T is the time between clock pulses. Equation (2-18) 
specifies R(t) over a time pT, which is one period. R(t) then repeats with 
this period. 

The power spectral density of R(t) in (2-18) can be obtained through 
the Wiener-Khintchine theorem. As shown in the appendix (section 2-5) (See 
also [9] for result, but not derivation): 


S(w) = 


1 

7 ~ 


6 (w) + 


P+1 


( 


sin w T/2 


wT/2 


2 <» 

l 

n=® 

n+0 


6 (w-n w Q ) 


(2-19) 


where vj - ~ . 

0 pT 

This spectrum is plotted in figure 2-4 for p=15. The first zero in the 

(sinx)/x envelope occurs at f = 1/T.That is, the zero of ■ s ~ - n — is dictated 

X 

by the pulse width or time between clocking pulses. The spacing between 
frequency components depends on the period. There are just p lines in 
the spectrum from the or.i gi n to the first zero of S(w). This result suggests 
that the pseudo-random signal can be used as an excellent source of repeatable 
"white" noise by decreasing T far enough so that the spectrum is essentially 
flat over the range of interest. 

To summarize: 

a) Decreasing T increases the frequency range over which the spectrum 
is "flat." 

b) Increasing p decreases the spacing between lines or increases 
the number of lines in a given band. 

For example, suppose we require a spectrum which is flat to 5%. 

What are the constraints on f and T ? 



8 10 12 14 16 18 20 22 24 26 28 


Figure 2-4. Power Spectral Density of a Pseudo-Random Signal 
Clock Frequency = — , n = 4 (p = 15) 
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wl 2 

/Sin 2\ = 0.95 

' wT } 

2 

wT 

sin = 0.975 

wT 
2 

so that 

fT * 0.125 
fmax = 


We now turn to the problem of the power spectral density and autocorrelation 
function of the output signal from the weighting system (or filter) of figure 0=1.. 
The corresponding input-output relations are [11]: 

V t) = R x ( t) * h{x) * h( " T) (2 “ 20) 

S y (w) = !H(jw)| 2 S x (w) (2-21) 

where H(jw) is the transfer function (or Fourier transform of the impulse 
response)of the system. 

It is probably easier in most cases to use equation (2-21). For the 
circuit of figure 0-1 the transfer function is: 


H(jw) = 


L 

4 , 


a^e 


-ijwT 


( 2 - 22 ) 


where T is again the time between clock pulses. Equation (2-22) is simply 
a series of shift terms obtain from the time-shift theorem of transform 
theory. Then: 


SyM ■ 


L 

U 


a -e 

l 


-ijwT 


S (w) 

X 


(2-23) 


At this point we alter the basic problem slightly, adding the input 
(with a weight ag) before it enters the register, to the other terms. S (w) is 

y 
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then: 


S v (w) - | l a i e- 3iwT | 2 S (w) 
y i=o x 


(2-24) 


Expansion of the j«j term leads to: 

s v (w) a i 2 * J * I l a,*a i+r cos rwT ] S (w) 

y i=0 ' r=l i=0 1 1 r x 


(2-25) 


If we let the input be a pseudo-random sequence of period p and clocking 

time T, then S (w) is given by (2-19) and S (w) becomes: 
x y 


v (w) = ?? i 0 a i" 5(w > + 


P+1 


sin ^ 


wT 

2 


2 °° 

l S (w-nw ) 
n=-” 
n^O 


L L-r 

X ( l l a. a, cos rwT) (2-26) 
r=l i=0 1 1 r 

For a particular set of{a.j }equation (2-26) may be reduced and eventually 
transformed to obtain Ry(x). Let us consider a special case where the input 
is pseudo-random and the weighting is uniform. A similar case where the weighting 
was uniform and the input purely random was solved by Wolf [12], and the 
development here closely parallels that work. 


If a,j=l for 0 £ i <_ L equation (2-22) becomes: 

H(jw) = e -3wLT/2 S1 - n g wi 

sin wT 
2 

L+l 

|H(jw)| 2 = (L+l) 2 sin 2 ~T~ wT 


sin 


2 wT 


(2-27) 


(2-28) 


(L+l r 


Substitution of (2-28) and (2-19) into (2-21) yields: 


s y (w) = (L+D 2 6 (w) + £+l_ (L+l) 2 - 
P 2 P 


sin (L+l) ^ 


(L+l) wT 
2 


(2-29) 

2 *> 

l 5 (w-nw Q ) 
n=«® 
nfO 
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A comparison of equations (2-29), (2-19) and (2-18) suggests that (2-29) 
must be the Fourier Transform of: 


R(t) - 


IT (L+1) (1 ' (l1]1 t > - t 1 -' 1 ' 1 ) 2 . |t| < (L+DT (2-30) 


-(L+1V 


(1+1 )T < |t| < (P-L-l ) T 


Again, R(t) is periodic with period pT. The result is simply another 
triangular form as shown in Figure 2-5, 


2.5 Appendix 

The purpose of this appendix is to show the derivation of equation (2-19) 
from (2-18). We start by writing equation (2-18) as: 

R(t) = R (t) - p- where (2-31) 


R (x) = 


P+1 

P 

0 


0-44 , m £t 


(2-32) 


, T < x < (P-l)T 


Since R(t) is periodic we seek' its Fourier series representation. 


2v\ 


R(0 * n L c " e 6 ’ <w ° = fP 


(2-33) 


The "DC" term, C Q , is: 


: 0 = pT } 


(P-l)T 


R(x)di 


- 1 f T P+1 n JLlL h 1 f U 
-pf ) — 0 “T } dT " pT J 
-T -T 


(P-l)T 


f dr 




PRECEDING PAGE BLANK NOT FILMED. 
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Z^y~ (2T - T) - 4“ pT 

rT P T 


The constant term - p- in R(r) v/ill not contribute to the "AC" co- 
efficients in the Fourier series, so we have: 


1 f (p-l)T 

C n = pT J R ' (x)dT 

-T 


= pT f tf- <1 - ¥-) 


2 I P+1 fi T\ J 

pT J p (1 " cosnw o TdT 

o 


since R'(t) is an even function. 


C - 2 < p+1 > ( 

L n 9 \ 


sin nw_T l-cosnw_T 


n Vi 


2(p+l ) 2 s1n nw o T/2 

" 2l 2 21 

p T n w. I 


2 21 
n w 0 T 

nw o T I 2 


= P + T sin 2 


sin nw„T 
o > 


We now substitute (2-34) and (2-35) into (2-33) to obtain: 


«x)-4 + i m 


nwT 1 2 . 


1 L 2 
p n=-“ p 


e jn V 


To find the power spectral density we take the Fourier transform of 


equation (2-36). 
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6( w) = f R(T)e" jWT dT (2-37) 

— 00 

This transform is obtained quite easily through use of the transform: 

[e jnw o T ] = 6(w - nw 6 ) (2-38) 

where 6 ( - ) is the impulse or Dirac function. Substitution of (2-36) 
into (2-37) and the application of (2-38) leads to: 

00 

1 s(w - nw Q ) (2-39) 

n=-« 0 

nj*o 
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3.0 The Probability Distributions of 
Certain Sums of Random Variables 

This section deals with sums of random variables of the form 

co 

Ss l d k s k . (3-1)* 

k=l K 

i this expression d is an arbitrary fixed rational number of the form ^ 
with g a natural number { 1 , 2, 3, 4, . . . and < s k *> is a sequence 
of independent, identically distributed random variables taking certain 
non-negative integer values { 0, 1, 2, 3, . . . } with equal probabilities. 

lie shall determine the distribution of the random variable S in the 
following four cases: 

Case (i) : S = Y ; d = 1 ; s R = y k = 0, 1 . 

Case (i i ) : S = 1 ; d = j ; s k = z k = 0, 1 9 2 . 

Case (iii): S ; d = , g a number from the set { 1, 2, 3, . . . 

^ Oj 2, ♦ » 9 ~ 1 • 

Case (iv) : S = X ; d = j ; s k = x R = 0, 2 . 

Cases (i) and (ii) are specializations of case (iii) . We shall use case 
(i) to illustrate the application of elementary probability theory to the 
determination of the (cumulative) distribution function of the sum S 
in case (iii) . We will see that the random variable S in cases (i), 

* The symbol s means "is defined to be." 
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(ii), and (iii ) is uniformly distributed on the closed unit interval [ 0, 1]* 
The main emphasis is on case (iv) in 'Which S = X has the Cantor distribution. 
Figure 3-1 gives an idea of the appearance of the Cantor function. We believe 
that some of our results in case (iv), especially the discrete approximations 
to the cumulative distribution function of the random variable X, are new and 
may be useful for computer study of sums of random variables of the form (3-1) 

3.1 The Uniform Cases 


Case (i) deals with the random variable 



(3-2) 


where y k * 0 or 1 , each with probability ^ , and the < y^ > are mutually 
independent. There are at least three methods for obtaining the (cumulative) 
distribution function F(v) = Prob { Y _< v } . The first method draws on 
combinatorial analysis, and it applies to all sums of random variables of the 
form (3-1). The second method uses convolution and leads to a functional 
equation for the cumulative distribution function F of the random variable 
Y. The third method employs the familiar transform technique (characteristic 
functions). We prefer the first method over the other two methods because 
of its intuitive appeal, and we will present the first method in this section. 
The other two methods appear in Cramer [13] . 

Let us determine F by the first method. We note that, for any 

m = 1 , 2 , 3 , . . . , 2 n and n = 1 , 2 , ...» the event { < Y < } 

2 n ~ 2 n 


* This notation is standard in mathematical literature. Thus it means 
here that the values of the random variable S lie in the interval 
' from 0 to 1 . . 
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occurs if and only if the first n random variables < y^ : k = 1 , 2 , ... , n > 
take a unique sequence of values < : k * 1 » 2, ... , n >, namely the first 


n digits in the binary expansion* of 


m-1 


Since the < y k > take the two 


1 


values 0 and 1 , each with probability j » anc * S1 ' nce the < y^ > are 
independent, the probability that the <y^:k=l,2, ... , n > take the 
particular values < a. : k = 1 , 2, . . . , n > is . Thus the probability 

K oU “v 

1 


that Y takes a value in an interval of the form 


> n % 
‘ r- m-1 m \ 

2 n * 2 n / 


equals 


which is the length of that interval. Now for each m and n , the events 

{ < Y < } are mutually exclusive. This implies that, for r = 1, 2, 3, 

2 n 2 n 

n ** 

..., 2 and n - 1, 2, ... , we have 


F(~ - } = Prob \ U { 
2' 


r 

U 

m=l 


fcl< v < SL- 

2 n - 2 n 


* l Prob ( < y < !2- } 

m =l 2 n “ 2 n 


m 1 

If the number e [ 0, 1) admits a binary expansion that terminates 
2 n 

after a finite number of digits, then there are actually two different 

m 1 

sequences < a. > of digits representing that number ~ . One sequence 
k 2** 

terminates after a finite number K of non-zero a^'s . The other 


sequence has repeating 1 1 s , i.e., a^ “ a (/ + 2 = = 1 > and it is usually 

excluded to ensure uniqueness of expansion (Kac [19]). In our problem, 
however, sequences containing repeating 1 1 s form a set of probability zero 
and so can be ignored. 

** The strict inequality Y < necessitates the minus sign in the argument 


of F because we defined F(v) = Prob { Y < v } rather than F(v) = Prob {Y < v} 
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Since the function G defined by G(v) = F(v-) is left-continuous on 

(0, 1] , and since for any v e (0, 1] there is a sequence of numbers of 

the form <~ : r = 1, 2, 3, ... , 2 n ; n = 1 , 2, converging to v 
2 n 

from below, it follows that F(v-) = v for every v e (0, 1] . Thus 

the distribution of Y is the uniform distribution on [0, 1] with 
(probability) density function F‘(v)=f(v) = 1, 0 < v < 1 . 

Case (ii) deals with the random variable 


2 s bli z k 


-k=i 3 


(3-3) 


where Zj, = 0, 1, or 2, each with probability -j , and the< z^> are mutually 

independent. To determine the cumulative distribution function 

F(v) h Prob {Z < v} , we can proceed as in case (i). Letting replace 

3 n 

— _ throughout, v/e can show that the probability of Z taking a value in 
2 -1 1 

an interval of the form [2^1 t 2L) equals — - , which is the lenqth of the 

3 3 n 3 

interval. The conclusion of the indicated procedure would be that the random 
variable Z has the cumulative distribution function F(v) = v for v e [0, 1], 
We remark in passing that expression (3-3) corresponds to a ternary expansion 
in which the digits <z^> have been made random variables. 

The preceding idea is easily generalized to case (iii) in which we have 
the random variable 


S E l d K S k . 
k=l k 


(3-4) 


* Read: The half-open unit interval that is open on the left, v=0, 

and closed on the right, v = 1. 
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_ 1 

The constant d may be any rational number of the form — with g 

0 

a natural number {1, 2, 3, 4, . ..} ; and <s R > is a sequence of independ- 
ent, identically distributed random variables -taking the values 
(0, 1, 2, ... , g-1) with equal probabilities , Considerations simi- 
lar to those preceding make it evident that expression (3-4) can be re- 
garded as the expansion of any number in the unit interval in the base 
g = ■g- . Thus the random variable S of expression (3-4) is uniformly 
distributed on [0, 1] . In summary, the random variable S of expres- 
sions (3-1) or (3-4) is uniformly distributed when the following two con- 
ditions hold simultaneously: 

1. < s R > mutually independent. 

2. Prob {s k = 0} = Prob {s R = 1} « ... = Prob {s R = ^ - 1} = d . 


3.2 A Non-uniform Case 


Case (iv) deals with the random variable 


03 

x- .iA* 


k=l 3 


k k 


(3-5) 


where x R = 0 or 2 , each with probability ~ , and the <x R > are mutually 

independent. There are at least two methods for arriving at a formula for 

the (cumulative) distribution function F(v) = Prob {X <_ v) of the random 

variable X. The first method uses a great deal of intuition. Motivated 

by Figure 3.1 that shows the cumulative distribution function F^ of the 

partial sum ) -r x,, » we speculate that the Cantor function is the cumu- 
k=l dr K 
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lative distribution function of the random variable X . Then we prove 
that this is indeed correct by using a theorem that we have formulated ex- 
pressly for this purpose. The second method is independent of the first 

method, and it uses a combinatorial argument. We determine a formula for 

K , 

the cumulative distribution functions F/,,\ of the partial sums Y x, , 

UW k=l 3 k K 

and then we obtain F as the limit of as K approaches infinity. 

The first method is quicker, and we will present it in this section. The 
second method appears in Cramer [13]; here we will give only the key results 
obtained from the second method. 

The Cantor function 6, which P. Halmos [14] gives, has the following 

form; 


V bu i_ 

k=l 2 k 2 L 


G(v) = | 


L(v) 


< 03 


(3-6) 


k=l JT » L(v) 


where 



b k = 0, 1, or 2; k = 1, 2, ..., 
. b k 

Cj, = g * k = 1, 2, . . . , L - 1 , 


and 
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L(v) = 


min { k: = 1 , k = 1, 2, ... > 

« if b k f 1, k = 1, 2, .... 

k. 


To show that G of equation (3-6) is the cumulative distribution function F, 
restricted to [0, 1] , of the random variable X, we have formulated the 
following: 


Theorem: 

If B is any non-decreasing function defined on the real line, 
taking values in the interval [0, 1], if X is any random variable, 
and if the random variable Y = B(X) has a uniform distribution 
on [0, 1], then B is continuous and is the (cumulative) distri- 
bution function of X . 


The proof of this theorem appears in the appendix. We now apply this theorem 
to the function G of equation (3-6) . P. Halmos [14] informs us that the 
Cantor function G has the following properties: G is non-decreasing, takes 
values between 0 and 1 , and is continuous. To see that G(X) has a uniform 
distribution on [0, 1], we consider 


m-.ill V.) = I \ • 

k=l 3 k k=l 2 K 1 


*k ~ 

Setting -jr s y^ , it is evident that the above expression is identical with 


expression (3-2) for the random variable Y which we have already shown to 
be uniform on [0, 1] in section 3.1. We therefore conclude that the random 
variable X of expression (3-5) has the Cantor function (3-6) as its 
cumulative distribution function. 
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Let us now outline the second method of obtaining the cumulative dis- 
tribution function F of the random variable X defined in expression 
(3-5) where we use a combinatorial argument. We write expression (3-5) in 
the form 


X 


lim 
K -> «> 




x 


k 


= lim 
K + 


l (K) 


where we define the partial sums 

K 


V) 5 tk 7 ■ 


(3-7) 


To determine formulas for the cumulative distribution function P{j/j( v ) = 

K 

Prob { _< v } , where K infinite, we first note that has 2 

jump discontinuities, each of size . (Figure 3.1 shows F^j(v) for 

K = 4.) If we count the number N^(v) of jump discontinuities that 
has in the interval [0, v], to the left of some given point v, then we 
have immediately 

(3-8) 


f (k) ( v) = pc N (K)< v) • 


Thus the determination of F^(v) reduces to the combinatorial problem of 
finding N^j(v) . Figure 3.1 suggests that it would be advantageous to 
express any given real number v in the domain of in its ternary 
expansion 

« b. 

v = I -jr ♦ 
k=l 3 k 
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The elements of the sequence <' b^> are numbers (not random variables), and 
they can be 0, 1, or 2. This representation of points v allows us to 

analyze the effect of each digit bj, on the number N^j,j(v) of jump dis- 

continuities in [0, v]. The work which we are leaving out here can be found 
in Cramer [13] ; let us state only the key results. The number k) ( v of 
jump discontinuities which has in the half-open interval [0, v) depends 

only on the digits <b^ : k = 1, 2, ...» min (L, K) > , i.e.. 


N( K) (v -) = N (k) 


min(L,K) b k 

1 — V ~ 

k=l 3 k 


The digits following ^ merely place v somewhat to the right of the 

min(L,K) b. 

point 7 — I- , but still within an interval on which F/,m is constant. 

k=l 3 k w 

The results of the combinatorial argument are the formulas 

k 


N (k) (v) = i 


c k + 1 , L(v) > K 


K K 

l 2 

k=l 

l f 2 K - k c k + 2 K “ L , L(v) < K 
k=l K 


(3-9) 


and the recursion formulas 


N (K + 1)M 


'2 N (k) (v) , L(v) < K + 1 
{2 N {k) (v) - 1 , L(v) = K + 1 
2 N^(v) - 1 + c K + -j , L(v) > K + 1 


b|r + 1 

where Cj, + -j = — ^ — = 0 or 1 when b^ + ^ = 0 or 2 . Equation (3-9) 
can be written in the equivalent form 



41 


min(L-l ,K) K . k K - min(L, K) 

N (K) (V) = k l } 2 C k + 2 


Substituting this expression into equation (3-8) yields the cumulative 
distribution function 

min(L-l,K) c 


F (K)< v) = -I 


1 


k=1 2 k + 2 min ^ L ^ K ^ * 


K finite, of the random variables defined in expression (3-7). 

Let us now obtain the cumulative distribution function F of the 

random variable X defined in expression (3-5) as the limit of F^j as K 

approaches infinity. Since X is the pointwise limit of as K 

approaches infinity, X^j also converges to X in distribution so that we 

can write F(v) = lim F/.,\(v) at all points v where F is continuous, 
K -*- «> (Kj 

i .e. , everywhere as shown in Cramer [13]. Performing the limiting operation 
yields 



This equation is exactly the Cantor function (3-6) that we arrived at in 
the earlier part of this section. 


3.3 

This section has dealt with 
where the random variables <Sj,> 


Conclusion 

r k 

the distributions of the sums ) d s,. 

k=l K 

are independent and identically distributed, 


each taking certain non-negative integer values {0, 1, 2, 3, ...} with equal 



42 


probabilities. Using elementary probability theory, it was shown that 
these sums are uniformly distributed on [0, 1] when ^ is a natural number 
{1, 2, 3, 4, ...} and the <s k > take the values {0, 1 , 2, . . . , -j- - 1 } for 
k = 1, 2, each with probability d . The main emphasis was on the sum 

co -j 

X = l -y x k where = 0 or 2, each with probability ^ . Two methods 
k=l 3 

were presented to show that X has the Cantor function as its cumulative 

distribution function F. The first method employed a theorem that, was 

formulated expressly to prove this. The second method used combinatorial 

analysis to arrive at formulas for the cumulative distribution functions 

K , 

of the partial sums I "X x k * ^ en ^ was °^ a ^ nec * as 

k-1 3 

of as K approaches infinity. 

The authors feel that this work may furnish a theoretical basis for 
further studies in the following areas of application: 

1. Output distributions of digital filters with known input processes. 

2. Distributions of the analogue sums of the weighted outputs obtained 
from feedback shift registers. 

3. Discrete systems identification using known discrete input processes. 

4. Singular detection and estimation problems. 

Co 

I 

5. Determination of the distributions of the sums T as,, where 

k=l K 

7 « 0 «’ • 

3.4 Appendix 


Let us prove the theorem that we stated in section 3.2. Since B is 

non-decreasing, X <_v implies B(X) _< B(v) . This means {X : X <_v} c 
{X : B(X) < B(v)> . Define B(X) = Y . Since Y is assumed to be uniform. 
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we have Prob {X £ v} £ Prob {Y £B(v)> = B(v} , Similarly, X >_ v implies 
B(X) >_ B(v) , so that Prob (X >_ v} £ Prob {Y >_ B(v)} = 1 - B(v) ; then 
1 - Prob ,{X < v} <_ 1 - B(v), and Prob {X < v} £ B(v) . We have thus obtained 
the inequalities 

B(v) £ Prob {X < v} £ Prob (X £ v} £ B(v) 
which lead to the equalities 

Prob {X < v} = Prob {X £ v> = B(v) . 

These equalities allow two conclusions. From the equality Prob { X £ v } = B(v) 
we conclude that B is the cumulative distribution function of the random 
variable X . And from the equality Prob {X < v} = Prob {X £ v} we conclude 
that X has no mass points, i.e., B is continuous. This completes the 
proof of the theorem. 




0 2_ 2 8 2 20 8 26 2 56 20 62 8 7* 2680, 

81 27 FT 9 FT 27 FT FFT 27 FFTFTFT ^ 

4 ^ 


Figure 3-1 .Cumulative Distribution Function '^( 4 ) (v) 
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4.0 Pseudo-Random Noise Generation 
And Digital Filter Implementation 

The purpose of this chapter is to discuss the hardware implementation 
of some of the devices used in previous chapters, and to describe some actual 
circuits built and some of the experimental results. 

A pseudo-random sequence of length L can be generated from a shift 
register containing n stages, where each stage can assume M different 
levels. With proper feedback connections, the length L can reach a maximum 
of M n -1 before repeating itself. In this chapter we describe the shift 
register, its use in generating pseudo-random sequences, the nonrecursive 
digital filter into which the sequence is fed, and the implementation of the 
noise generator and filter. 


4.1 Shift Register 

Let us consider the n-stage shift register shown in Figure 4-1. Each 
stage can assume the values 0 or 1. Two inputs are provided to- the register: 
a clock input (CP) and a data input. 

When the clock pulse input is activated, each stage assumes the state 
of the stage on its left. The first stage assumes the state of the data 
input. 

A hardware realization of a binary shift register uses flip-flops as 

its constituent stages. The output of a flip-flop can assume one of two 

levels, the logical 0 and 1. We will assume that the hardware realization 

uses 0-K flip-flops, whose characteristic table and logic diagram are given 
K 

in Figure 4-2. Q represents the output at the kth clock pulse, Q the 

-k k 

output at the (k+l)th clock pulse, and Q the complement of Q . The logic 
diagram representing Figure 4-1 will then be as shown in Figure 4-3, where the 
sumbol-()o- represents an inverter. 
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4.2 Pseudo-Random Sequence Generator 


The sequence of states of any of the flip-flops of the register shown 
in Figure 4-3 is a maximal -length pseudo-random sequence if the proper data 
are fed into the first flip-flop. These data can be generated by a feedback 
configuration, involving two or more connections as shown in Figure 4-4, 
where -e-^e-b denotes a modulo-2 adder, with truth table 



0 

1 

0 

0 

1 

1 

1 

0 



The symbols used here are defined by their truth table given in Figure 4-5. 

The maximal length sequence L = 2 n - 1 will be achieved before repetition of 
the sequence given the proper feedback connections. For certain lengths of the 
register, feedback from the output of only two stages will not give the maximal 
length, and more than two feedback connections are required. Table 4-1 gives the 
possible feedback connections for a maximal length sequence when the number n 
of stages goes from 4 to 15. A maximal -length four-stage pseudo-random sequence 
can then be described by Figure 4-6. The states of the flip-flops of Figure 4-6 
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Representation of a Pseudo-Random Sequence Generator 

Figure 4-4 



a) AND gate b) OR gate c) NAND gate 

Symbols and Truth Table of Logic Functions 
Figure 4-5 



Logic Diagram of a 4-stage Pseudo-Random Sequence Generator 

Figure 4-6 
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n 


V 

feedback connections for maximal length 2 '-1 


4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 


1 © 4 or 3 © 4 

2 © 5 or 3 © 5 

1 © 6 or 5 © 6 

1©7 or 3©7 or 4©7 or 6©7 

3 © 5 © 7 © 8 

4 © 9 or 4 © 9 

3 © 10 or 7 © 10 

2 © 11 or 9 © 11 
6 © 8 © 11 © 12 

4 © 6 © 10 © 13 
4 © 8 © 13 © 14 

4 © 15 or 7 « 15 or 8 © 15 or 14 © 15 


Feedback Connections 


Table 4-1 
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are shown in the timing chart of Figure 4-7, assuming that all the flip-flops 
have been set to 1 at t=0. Any one of the columns is a pseudo -random sequence 
of 0 and 1. It should be noted that the all -zero state of the register never 
occurs. If it did, the register would be locked in that state. 

4.3 Digital Filter 

Consider the preceding shift register with n stages, and a clock of 

frequency f Hertz. A shift will occur every T seconds (T = l/f„). At time 

kT, the last stage of the register contains the state of the first stage at 

time (kT - (n - 1 ) T) , or (k - n + 1 )T. At any given time the states of 

the first stage at times kT, kT-T, kT-2T, kT-3T, up to kT-(n-l)T are present 

in the register. This suggests the possibility of "filtering" the sequence 

using a nonrecursive digital filter defined by the equation: 

n-1 

y(kT) = l a. x (kT-iT) (4-1) 

i=0 1 

where y(kT) and x(kT) are the output' and input of the filter at time kT 
respectively, and a^ are the weights given to the present input and n-1 
previous inputs. With the representation of Figure 4-1, equation (4-1) can 
be realized by Figure 4-8. A number of hardware realizations of Figure 4-8 
can easily be imagined. The simplest one implements the weights by resistors, 
as shown in Figure 4-9. If we wish to make the contribution of the output of 
a stage to the total sum y(kT) independent of the state of the other stages 
(condition that has to be met to assure the linearity of the output summer), 
then this configuration limits us greatly in the choice of acceptable values 
for the Rj's. Any Rj should always be much larger than R, such that, looking 
from the output of one stage, R looks much smaller than the parallel combin- 
ation of all the other Rj's. Given R^»R, can be neglected, and, for all 















A Simple Hardware Realization of Figure 4-8 
Figure 4-9 



Figure 4-10 
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practical purposes, the current through R is the sum of the currents 
through each Rj. This is shown in Figure 4-10. Ideally, we would not like 
to be limited in the range of available weights. Unless v/e use active devices, 
we cannot expect the weights to exceed 1, and the above realization adds a 
further limitation on the lowest acceptable v/eight. 

Multipliers can be implemented using operational amplifiers, represented 
by the symbol — , as shown in Figure 4-11 where V QUt = Vin 


This last factor is the weight a. of the filter, having a range that is limited 

J 

only by the operating characteristics of the amplifier. Figure 4-8 would then 
have the realization of Figure 4-12. The condition ^>>^2 still holds, but 
does not have any effect on the weights defined by the input and feedback 
resistors of the operational amplifier. 

A simpler realization of Figure 4-8 can be implemented using a single ' 
operational amplifier, in a sunnier configuration. Consider the operational 
amplifier of Figure 4-13 with two inputs (inverted and non-inverted). All 
resistors have the same value. Due to the non-inverted input held at ground 
level, point 0 can be considered very close to ground level, independent 
of the input and feedback currents. Since the input impedance is very high 
(of the order of megaohms), the current into the amplifier can be neglected, 
and thus 


\ = l R* \ 


+ Ir 


(4-2) 


Since all resistors have the same value, the output voltage is 


V o ■ V 1 * V 2 + V 3 


(4-3) 


If v/e want to add weighting factors to the different inputs,- the values of 
the input resistors can be varied to give 
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R 1 

R f 
V l ( R i) 


1 



Rf Rf 

+ Vr^ + Vr^ 


(4-4) 


where the factors R f /R. are the weighting factors. An alternate way of 
weighting the input currents is to weight the input voltages before sending 
them into the sunming circuit as shown in Figure 4-14. The current through 
the input resistor is negligible in comparison to the current through the 
variable resistor. 

This last configuration has been chosen in our implementation. 

Its shortcoming compared with the previous configuration is the limitation 
in the range of weighting factors (0 to 1), but its simplicity (1 operational 
amplifier against k) offsets the shortcoming. 

All essential elements for the realization of the pseudo-random sequence 
generator and the digital filter have been presented. A few more details have 
to be added. 

In the implementation of some nonrecursive digital filters, some weights 
assume a negative value. A resistor cannot have a negative value, but the 
voltage applied to it can be inverted, giving the same effect. In the 
implementation of a shift register using flip-flops, the output of any of 
the stages always has its complement available from the other output of the 
flip-flop. This is shown in Figure 4-15. 

One of the methods of finding the weights of the filter is to realize 
the inverse Fourier transform of the required frequency spectrum. The 
result is the impulse response of the filter. For final calibration of the 
weights, it is useful to see this impulse response on the screen of an 
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oscilloscope. A way of feeding an impulse to the filter must be provided. 

The sequence {1 ,0,0, 0,0, 0,0, ... , ^containing m terms (m larger than n, the 
number of stages used by the filter), is fed into the filter from the shift 
register and repeated to provide a continuous display on the oscilloscope. 

The reset line for the register sets the first stage, and clears all the 
other stages (at the same time providing for the initial conditions appropriate 
for the generation of the random sequence). The resistor should provide the 
option of a circular configuration, where the first stage assumes the state 
of the last stage when the CP input is activated. (This method is used in 
section 4-7. See Figure 4-32). 

The variable resistors of Figure 4-14 are calibrated to give the 
required weights. If the stage associated with the resistor to be calibrated 
is in the 1 state, with all the other stages in the 0 state, the output 
voltage of the filter will be a function of the setting of that particular 
variable resistor, and the weight will be given by 



where is the measured output voltage, and V max is the voltage chosen to 
represent the weight of 1 . A manual clock and a D.C. voltmeter at the output 
are provided to facilitate the calibration. 

When testing the operation of the shift register and when calibrating 
the resistors, it is useful to have a visual display of the state of the 
stages used by the filter. The output of the stages can be amplified and 
sent to a light bulb. The clock should have a frequency low enough to allow 
time to check the feedback operations and the shifting. 

Figure 4-16 shows a logic diagram of a four stages shift register 
together with a digital filter using all four stages. 
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4.4 A Hardware Real i zation_ of .the Random Sequence Generator 


Implementation of the register of Figure 4-16 uses Digital Equipment 

Corporation (DEC) flip chip modules. Reference [15] gives a detailed 

description of the modules. We will present here only the parts of the 

modules that are used in the implementation of the register. The modules 

are mounted on a DEC H901 mounting panel , with a type DEC 700D power supply 

and input panel that provides for the push button pulsers and a clock. 

Figure 4-17 shows the logic diagram for the DEC R201 flip-flop. 

Fifteen of these were used (implementation of a 15-stage shift register). 

The symbol— j><^j denotes a' diode-capacitor-diode (DCD) gateT " The* feedback" 
T 

logic is realized with the DEC Rill NAND/NOR gates, shown in Figure 4-18, 
with the following symbol. to represent a common emitter transistor: 


. Collector output 


Base inout 




.Emi tter 

The DEC R107, shown in Figure 4-19, is used to provide for the 
complement of some of the outputs. 

The DEC W520 .comparator and DEC W501 Schmitt trigger "were used to 
provide some means for applying an external clock signal and for feeding an 
external binary sequence. They are. shown in Figure 4-20. The symbol ~ ~ |^> 
stands for a difference amplifier. 

The complete wiring diagram for a 15-stage pseudo-random noise generator 
is given in Figure 4-21. THe outputs of stages 14 and 15 are used as feed- 
back. The dotted lines coming from the DEC RT07 module indicate alternate, 
connections when an external sequence is fed into the register. In this 
case, the feedback connections have to be disconnected (disconnect the wire 
going into S of R107 coming from Rill). 






.>.201 flip-flop 
Figure 4-17 


h n u 



Rl 1 1 


NMDA'OR Gates 



R107 Inverter 
Figure 4-19 
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a) W520 



b) W501 Schmitt trigger 


Figure 4-20 
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4.5 A Hardware Realization of a Nonrecursive Digital Filter 

The main difficulty encountered when implementing the diagram of 
Figure 4-14 was finding an operational amplifier with a good response at a 
clock frequency around 1 MHz. The Fairchild integrated circuit yA709 and 
its self-compensated version, the yA741 , were first used with a voltage gain 
of 10. With a slew rate of .3 V/sec. at unity gain, the output waveform was 
greatly distorted, making it difficult to operate at a clock frequency faster 
than lOOKHz. (The slew rate is one of the factors describing the operation 
of an operational amplifier: it is defined as the rate of change in the out- 
put voltage when a step input voltage saturates one of the. inputs). 

The Fairchild yA715 is designed for high-frequency applications, with 
a slew rate of 65 V/sec. at a voltage gain of 100, and 20 V/sec. at unity 
gain. Efforts were made to use this operational amplifier, but major 
difficulties were encountered when trying to compensate it. After repeated 
trials, the "ringing" at the output without an applied input signal could 
still not be eliminated. 

The Fairchild yA702C High Gain, Wideband DC Amplifier was chosen. It 
has a slew rate six times faster than the yA741 , giving satisfactory operating 
characteristics at a clock frequency of 10 MHz. 

In Figure 4-22 the input comes from the output (direct or complemented) 
of the first seven stages of the shift register of Figure 4-20. The 
positive voltage applied at pin 8 of the amplifier is provided by the 700D 
power supply, and a negative voltage of 7 volts coming from an external power 
supply is applied at pin 4. The characteristics of the yA702C are given in 
the appendix. 


4.6 Experimental Measurements on the Linearity and 
Frequency Response of the Digital Filter 
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A nonrecursive digital filter is defined by equation(4-l ) repeated 

here for convenience 
m-1 

y(kT) = l a. x(kT - iT) (4-1) 

i=0 ‘ 

The output y(kT) is a linear function of the actual input and the (m-1) 
previous inputs. The circuit of Figure 4-22 will realize equation' (4-1 )"'o‘nly~ 
if the operational amplifier has a linear characteristic in its voltage range 
of operation. The maximum output voltage is reached under the conditions 
a i = 1 

x(kT - iT) « 1 (logical) 

for all i's. It .is a function of the voltage gain of the feedback amplifier, 

* 

the voltage level associated with the logical state 1, and the number of 
stages used by the filter. 

To check the Vineartty of the feedback amplifier in its range of 
operation, all the weights can be set to 1, and a sequence of m 0's 
followed by m l l s can be fed into the shift register connected in a circular 
configuration , m stands for the number of shift-register stages used by 

% 

the filter. With a linear characteristic of the summer, the oscilloscope 

display should look like a staircase v/ith a constant increase b etween eac h 

step. The experimental results are shown in Figure 4-23. The results 
indicate excellent amplifier linearity. 

Another important characteristic of the summer of Figure 4-22 is a 
good frequency response at the clock frequency used when filtering input , 
signals. Ideally, for an applied step input, we would like the output to 
rise instantly without any overshoot. Practically we are limited by a 
definite slew rate and a certain amount of overshoot, generally larger as the 



not reproducible 
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Linear characteristics of the summer of figure 4-22 , 

Figure 4-23 

Clock period 333yS 

Oscilloscope sweep time .2mS/cm 
Sensitivity .2V/cm 

Weight setting a .=1 for all j's 

J 
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slew rate increases. The overshoot is eliminated by the output low- 
pass filter of Figure 4-22. Figure 4-24 shows the output of the four- 
stage pseudo-random sequence generator with clock periods of 3.33 us. and 
1 us., without output filter. Experiments have shown that such overshoot 
adds a few low frequency components to the power spectrum of the pseudo- 
random sequence, and high frequency components outside the range of interest. 
The time constant of the output filter has been found by trial and error. 
Varying the time constant by regular steps; the overshoot was reduced to 
a point where the low frequency components due to overshoot reached a minimum. 
The final shape of a four-stage pseudo-random sequence is shown in Figure 
4-25, at three different clock frequencies. From these results, we chose the 
clock frequency of 300 KHz. to run the experiments presented in the next 
section. It is the fastest clock frequency without serious distortion of 
the output. 

A close look at Figure 4-24 a will show a small fluctuation at each 
clock pulse when the output stays at the same level betv/een pulses. This is 
due to a very high frequency oscillation of the output of the stages of the 
shift register at the clock pulse, when the output should not change level. 
This could be eliminated by the use of a better flip-flop to realize the 
shift register, or by a lowpass filter inserted between the output of the 
stages and the weights of the filter. 

4.7 Design and Experimental Results 
of a Lowpass Digital Filter 

In this section, we will present a method for finding the weights of 
a nonrecursive digital filter with m delays, design a lowpass filter, 
describe the statistical characteristics of the input to the filter (a 15- 
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ampl itude 



b) clock period of 1 pS 



a) clock period of 3.33yS 


Output of a 4-Stage Pseudo-Random Sequence Generator Without Output Filter 

Figure 4-24 



amplitude 
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c ! ) clock period of 1 yS 


amplitude 
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<ampl itude 


b) clock period of 3.33yS 



time 


a) clock period of lOyS 

j 

Output of a 4-Stage Pseudo-Random Sequence Generator With Output Lowpass 

Filter 


Figure 4-25 
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stage pseudo-random sequence), and finally present the experimental results 
of the filter implementation. 

The frequency response S(wT) of a nonrecursive digital filter is 
given by 


Equation (4-6) is assumed periodic in wT and defined for -it < wT <_ ir . 

The B n 's are the Fourier coefficients of the periodic function S(wT), and 
are given by 


Given a filter specified in the frequency domain by S(wT), the inverse Fourier 
transform of S(wT) will give the B p ' s that describe the impulse response of 
the filter. As most periodic functions are exactly described only by an 
infinite number of Fourier harmonics, there will be, in most cases, an 
infinite number of B n 's. If we v/ant to realize the desired frequency 
response by a nonrecursive digital filter which has a finite impulse response 
(it has only a finite number of delays), the infinite series of B p ' s will 
have to be truncated, some B n 's made zero outside a given aperture. The 
implementation of a desired S(v;T) on a nonrecursive digital filter will then 
be only an approximation of an ideal frequency response. The larger the 
number of delays, the more accurate will be the implementation. 

To find the number of delays required for a desired accuracy, 
successive trials have to be made: first obtain the P. n 's by taking the inverse 
Fourier transform of S(wT); truncate the series of B n 's according to a chosen 
aperture, and transform the truncated series to get a modified S'(w T), the 


S(wT) - I e e- JnwT 

ns.m 


( 4 - 6 ) 



(4-7) 
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approximation to the ideal S(wT). If the modified S'(wT) is not accurate 
enough, try a different number of e n ' s (qo from m delays to m' delays), or 
change the position of the aperture, and repeat the process until 
results are obtained. (A weighting function, called a "window", can be used 
to modify the 6 p 1 s , improving the shape of S'(wT). Examples of these are 
the Hanning window, the Hamming window, the Blackman window, etc. The use 
of a weighting function is neither discussed nor applied here.) 

The Fast Fourier Transform (FFT) is of great help in determining the 

number of delays and the weights of the filter. The function S(wT) is put 

into sampled form, using M samples. For use with the FFT, the number M 

should be a pov/er of 2. We give an example of the method by realizing 

a lowpass filter with m = 7 (the number of delays in Figure 4-22). The input 

1 5 

to the filter will be a pseudo-random sequence of length 2 - 1, that has a 

power spectrum given by equation 2-19, and shown in Figure 4-26. (Figures 
4-27 and 4-28 are further illustrations of equation 2-19. They are the power 
spectra of the output of a four-stage generator, with two different scales.) 
We would like to filter out all the frequency components of the first lobe 
higher than f /3, with f the clock frequency of 300 KHz. As S(wT) is 

L L 

periodic for a digital filter, the desired frequency characteristics S(wT) 
of the filter will be as shown in Figure 4-29. Part of the power spectrum 
of the first lobe of the shape Sin x / x input is shown in Figure 4-30. 

The cutoff frequency of the desired filter is indicated bv f . The frequency 
response of the ideal lowpass filter for -x <_ wT <_ x is shown in Figure 
4-31 a ) in sampled form; its Inverse Fast Fourier Transform (IFFT) is given 
by b) of the same figure. As the hardware implementation has only 7 stages 
(Figure 4-22), we chose an aperture of 7 aT (when aT = 1 / f c ) , centered about 
B 0 . Making all the other e n 's zero (Figure 4-31 c) ) and taking the FFT 
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Power Spectrum of the Output of a 15-Stage Pseudo-Random Sequence Generator 


Figure 4-26 
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Power Spectrum of a Four-Stage Pseudo-Random Sequence Qenerator 


Figure 4-27 
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Power Spectrum of a 4-Stage Pseudo-Random Sequence Generator 

Figure 4-28 
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Figure 4-30 
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d) Approximated frequence response 




a) Desired frequency response * 


Steps for finding the fi 's of the filter 

n 


Figure 4-31 
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of the truncated series, we get the frequency characteristics of the filter 
shown in Figure 4-31 d), which is an approximation of the ideal character- 
istics of a) in the same figure. Assuming that this approximation is accurate 
enough for our purpose, we should realize this filter by implementing the 
following weights: 

a 1 = -.194 
a 2 = -.059 
a 3 = .597 

a 4 = 1*0 

a 5 = .597 

ag = -.059 
a 7 = -.194 

After setting of the variable resistors of Figure 4-22, the impulse 
response of the filter can be checked by circulating a 1 in the shift 
register, all other stages being in the state 0. This impulse response is 
shown in Figure 4-32. The approximation to a - x form is apparent. 

The power spectrum of the digitally-filtered pseudo-random sequence is 
shown in Figure 4-33. The scaling is identical to the one in Figure 4-28, 

which is the input to the filter. Figure 4-34 gives a better idea of the 

periodicity of a digital filter. It shows the output of the filter for 
about 5 cycles. The input to the filter corresponding to the same is 

shown in Figure 4-26. For curiosity, we have shown in Figure 4-35 the output 

of the filter in the time domain for a short part of the long periodic sequence. 

4.8 Software Simulation of a Lowpass Digital Filter 

Finally, we present a simulation on the IBM 1130 of the lowpass 
nonrecursive digital filter presented above. 
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Impulse response of the lowpass nonrecursive digital filter 


Figure 4-32 
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Power spectrum of the filtered output of a lB-staoe pseudo- random sequence 
generator 

Figure 4-34 
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Time response of the lowpass nonrecursive digital filter 


Figure 4-35 
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A Fortran program has been written containing three main parts: 
generation of the input pseudo-random sequence, simulation of the filter, 
and Fast Fourier Transform of the output. Because of the limitation in the 
computer memory size available (16,000 words of 16 bits), we could not 
simulate the complete output of a 15-stage noise generator. Instead we 
used a 9-stage shift register (with states .5 or -.5 to avoid D.C. component 
in the power spectrum), with a sequence of 512 terms. The change in the 
number of stages used has the effect of increasing the distance between the 
power spectral lines, without affecting the sin x/x envelope. The power 
spectrum of the input is shown in Figure 4-36. 

The output of the filtered sequence is shown in Figure 4-37. This 
simulated result agrees very closely v/i th the experimental results. 
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Figure 4-37 
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Spectrum of the output of the filter 


5.0 Generation of Partition Numbers 
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This section describes an interesting and unexpected approach to the 
generation of partition numbers and some other sequences through the use 
of convolution, and digital filtering techniqes. 

5.1 Partition Numbers and Convolution 

A partition of a positive integer is the expression of the integer as 
a sum of positive integers. For example, the integer 4 has 5 partitions. 

4 = 4 
4 = 3 + 1 

4 = 2 + 2 

4 = 2 + 1 +1 

4 = 1 + 1 + 1 + 1 

A change in order is not considered to lead to a new partition. The 
number of partitions of the integer n shall be called the "partition 
number" here, and denoted p(n). From the above example, p(4) = 5. 

It is well known (see for example, Alder [16]) that a generating function 
for p(n) is given by 

f(x) - = . I x I < 1 (5-1) 

n (1 - x v ) 

V=1 

An alternative to this multiplication procedure for obtaining p(n) is 
suggested by the convolution theorem of operational calculus. 

Consider the following expansion of equation (5-1) 
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= (1 + x + x 2 + , : .) • (1 + x 2 + x 4 + • • •) 

•(1 + x 3 + x 6 + • * •) • • • (5-2) 

The p(n) are the coefficients of the x n terms in the infinite product. The 
sums in parentheses in equation (5-2) are each in the form of a z-transform 
of a sequence. The sequences corresponding to the first three sums are: 

{ 1 1 1 1 1 1 ♦ • • } 

{ 1 0 1 0 1 0 • •• • } 

{ 1 0 0 1 0 0 • • ♦ } 


Further terms have associated sequences with progressively more 
zeros between ones. Since equation (5-2) is.-essentially a product of 
z- transforms, we know that f(x) is the z-transform of a sequence which is 
the convolution of the sequences associated with the sums on the right in 
(5-2). This follows from the convolution theorem. That is: 

F(«) ={1 1 1 1 1 1 * * - }*T1 01 01 01 • * *>* 

{1001 001 0 * * • } * • • • 

where F(w) is the z-transform of f(x) and * indicates discrete convolution 
(See Healy [18]) . to is the set of positive integers associated with the - 
terms in the sequences, in ascending order from 0. At first glance it might 
seem that an infinite numbers of convolutions need be carried out to obtain 
values of P(n), which are just the terms in F(u>), But actually, since the 
k tn sequence has the form: 
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{ 100 * * * - 00100 * * * •> 

- y 

k - 1 

convolution by this and higher order sequences will reproduce the first k 
terms in Ffto) without change. Hence to obtain the first k terns in F(w), 
that is the first k numbers of partitions p(n), it is only necessary to 
convolve the first k sequences. Fortunately, this operation is easily 
done in tabular form as shown in table 5-1. The table is carried out far 
enough to -generate the first 9 values of p(n), which appear as the first 

x L 

nine numbers in the 9 (bottom) row. 

t t 

t 

The table is generated in the following way. Write a sequence of ones 
as the top line. To get a number in the next row, add the number above the 
desired number to the second number to the lefc plus the fourth number to 
the left, etc. until you reach the left. To get any number in the j tn row, 
add the number above it in the j-1 row, to the number j units to the left in 
the j-1 row, to the number 2j units to the left, etc. until you reach the 
left. (e. g. in the third row, 12=5+4+2+1). The above procedure 
simply carries out the process of successive discrete convolution. 

Table-5-1 has some additional information. Consider the diagonal rows 

-.L 

(+ 45° } . For example, the 6 diagonal row from the upper left is 
{133211}, reading from upper right to lower left. The sequence 
gives the number of partitions of 6 into u parts where y is the order of the 

"fc H 

numbers in the sequence. Similarly, the m diagonal row gives the number 
of partitions of m into y ■ parts as a sequence of values corresponding .to 
y = 1, 2, 3, . The number of partitions of m into parts the largest 

of which is y is of course given by the inverse sequence as suggested by 
Alder's E 1 63 theorem 2. A similar procedure can be used to find numbers of 
partitions into even parts or odd parts or a number of other possible forms. 
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5.2 Digital Filter Generation 

The above convolution of sequences suggests another viewpoint, or 
method of generating p(n). The sequences of ones, separated by k zeros, 
is simply the discrete impulse [ { 1 0 0 0 • • • > for discrete 
systems] response of a one feedback-stage recursive digital' filter with 
+ 1 units of delay. This is illustrated in figure 5-1. Hence the 
necessary convolution of sequences with increasing zeros-spacing (value 
of k) can be obtained as the discrete impulse response of a cascade of 
filters with increasing k, as shown in figure 5-2. 

The output of the first stage in figure 5-2 is the first line in 
table 5-1, the output of the second stage is the second line, etc. It is 
apparent from the argument in the_ preceding section that to obtain _the_ 
first k p(n) it is necessary to cascade k filters. 

5.3 Generation of Other Sequences 

1 1 should be apparent from the above that there are man., sequences 

which can be generated by digital filters. One example is the Fibonacci 
sequence in which each number is the sum of the previous two numbers in 
the sequence. The sequence begins as: 

1 1 2 3 5 8 13 21 34 55 * * • 

This sequence has a generating function: 

G(x) " 1 (5-3) 

1 - x - x 2 

From the fundamental property of the sequence, as expressed above, or 
from the generating function, we deduce that the sequence can be generated 
by a two stage recursive digital filter as shown in figure 5-3. This 




{1 1 2 2 3 3 — } { 1 1 2 3 5 6 9 ♦•♦ > 



Figure 5-2. Digital Filter Partition Number Generator 


to 
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1 1 1 1 1 1 1 1 1 1 • * * 

1122334455* * * 

1 1 2 3 4 5 7 8 10 12 • • • 

1 1 2 3 5 6 9 11 15 18 • • • 

1 1 2 3 5 7 10 13 18 23 • • • 

1 1 2 3 5 7 11 14 20 26 * * * 

1 1 2 3 5 7 11 15 21 28 * * * 

1 1 2 3 5 7 11 15 22 29 * * * 

1 1 2 3 5 7 11 15 22 30 * * * 


Table 5-1 
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circuit adds the number delayed by one time period (Z~^) to the number delayed 

O 

by two time periods (Z ) to form the new output. 

It is apparent that many other sequences can be generated by other 
feedback schemes. 
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